Asymptotical Properties of Residual Bootstrap for Autoregressions

In this paper we deal with stationary autoregressive processes of nite or in nite but unknown order. Under fairly general assumptions we derive the asymptotic consistency of a usual residual bootstrap procedure for smooth functions of the empirical autocovariance and autocorrelation. Especially the order of the tted autoregressive model is allowed to be data-dependent. Supplementary to the usual residual bootstrap we consider a wild bootstrap procedure. Some remarks concerning the asymptotic accuracy of the two proposed bootstrap procedures and a simulation study conclude the paper. 1. Introduction Let us consider the following strictly stationary univariate autoregressive process X = (Xt : t 2 Z = f0; 1; 2; . . .g) which full lls the following stochastic di erence equation Xt = 1 X =1 a Xt + "t ; t 2 Z ; (1.1) where we assume for the white noise process " = ("t : t 2 Z ) (A1) ("t : t 2 Z ) consists of independent and identically distributed (i.i.d.) random variables with E"t = 0 and Var "t = E"2t = 2 > 0 : F denotes the (cumulative) distribution function of L("1) : In order to obtain stationary solutions of (1.1) we assume for the parameter # = (a : 2 IN = f1; 2; . . .g) (A2) P1 =1 ja j <1 and 1 P1 =1 a z 6= 0 for all complex jzj 1 : The intention of this paper is to investigate asymptotical properties of residual-based bootstrap procedures for estimators based on empirical autocovariances, i.e. ̂T (h) = 1 T T h Xt=1 XtXt+h ; h = 0; 1; 2; . . . : (1.2) This class includes estimators for the theoretical autocorrelation, for the parameters of a tted autoregression of given order q, say, for autoregressive spectral density estimates and smoothed periodogram estimators. The main idea is to t an autoregressive scheme of order P to given observationsX1; . . . ;XT according to (1.1). The order P may depend on the sample size T as well as on the data themselves. Such an autoregessive t supplies us with estimated residuals "̂t = Xt PP =1 â (P )Xt : Denote by F̂ c T the corresponding centered empirical distribution function of the estimated residuals (cf. Section 2 for details). The usual residual bootstrap procedure is now de ned as an autoregression with order P , parameters â1(P ); . . . ; âP (P ) and i.i.d. innovations " t distributed according to F̂ c T : This usual residual bootstrap turns out to be asymptotically consistent for estimators based on the empirical autocovariance as well as on the empirical autocorrelation function. The idea of bootstrapping AR(1)-processes was introduced in Kreiss (1988). It has been successfully applied to ARMA-processes of nite order (recall that invertible ARMAprocesses have an autoregressive representation), cf. Kreiss and Franke (1992). Swanepoel and van Wyk (1986) applied this idea to spectral density estimates. In contrast to their paper we give a reasonable amount of theory for this speci c situation in Section 6. Paparoditis and Streitberg (1992) applied the AR(1)-bootstrap to order selection procedures in multivariate ARMA-models. 1 Recently B uhlmann (1995) considers this bootstrap proposal and compares it with the so-called blockwise bootstrap which was introduced by K unsch (1989). See also B uhlmann (1993) and B uhlmann and K unsch (1994). Bickel and B uhlmann (1995) consider in detail the probabilistic structure of this kind bootstrap. Especially, they obtain a new kind of mixing property for the bootstrap process. In contrast to all mentioned papers we allow that the order of the tted autoregressive model may depend on the data, i.e. it may be obtained from a usual order selection procedure. Moreover we consider di erent kinds of resampling schemes for our setup. Franke and Hardle (1992) and Dahlhaus and Janas (1996) consider a completely di erent bootstrap approach for time series data, which is based on the periodogram. This approach yields consistent bootstrap approximations for the distribution of statistics which can be represented as functionals of the periodogram. K unsch (1989), B uhlmann (1993) and B uhlmann and K unsch (1994) proposed the socalled blockwise-bootstrap for dependent observations. This approach does not need an autoregressive scheme for the data-generating process and is in this respect more exible. On the other hand, if the underlying structure is of autoregressive type (including all invertible ARMA-structures) the proposal given in this paper is much easier to apply and theoretically much easier to handle. Wu (1986) introduced the idea of wild bootstrap. In the context of autoregressive models it means to replace the complicated process structure of the data by a much simpler regression model with (conditionally) xed design in the bootstrap world. In this case it su ces to use independent innovations "yt distributed such that E "yt = 0 and E "yt k = "̂kt (k = 2; 3) : The situations in which this much simpler wild bootstrap works for autoregressions are more restricted and will be discussed in detail in Section 4. The paper is organized as follows. Section 2 contains exact de nitions of the bootstrap proposals which will be investigated in the paper. Asymptotic properties of the usual residual bootstrap are given in Section 3, while Section 4 is devoted to the wild bootstrap proposal. Some remarks concerning the accuracy of the bootstrap proposals will be given in Section 5. Section 6 deals with bootstrapping spectral density estimates. Using results from Section 3 and 4, we are able to show asymptotic consistency for such a proposal. Simulation results, which will be reported on in Section 7, demonstrate the nite sample properties of the bootstrap proposals for various estimators. Finally, Section 8 contains most of the proofs and some necessary auxiliary results. 2. The Bootstrap Procedures Suppose that observations X1 pmax(T ); . . . ;XT (pmax(T ) is de ned below) of the underlying autoregressive process (1.1) are available. At rst we t an autoregression of order P (T ) P to the data. P may depend on the data, e.g. the order P may be obtained from a usual 2 order selection criterion (AIC, BIC, Rissanen-Schwartz criterion and so on). To be as exible as possible we merely assume throughout the whole paper that (A3) P P (T ) 2 [pmin(T ); pmax(T )] ; where the deterministic sequences pmin(T ) and pmax(T ) are assumed to full ll pmax(T ) pmin(T )!1 and pmax(T )7 (log T )2 =T 2 ! 0 as T !1 : Remark. The assumptions on pmin(T ) and pmax(T ) are in full strength not really necessary for all asymptotic results. But we nd it much more convenient to work with a single assumption on pmin(T ) and pmax(T ) throughout the whole paper. If for example X is an autoregression of nite order po, say, (that is a = 0 for all > po) then it su ces that lim infT!1 pmin(T ) po holds. Using the autoregressive t of order P introduced above, we easily obtain estimates "̂t = "̂t(P ) of the innovations "t according to "̂t = Xt P X =1 â (P )Xt ; t = 1; . . . ; T : (2.1) Here and in the following we use the parameter estimator #̂T (P ) = (â1(P ); . . . ; âP (P ))T ; (2.2) which is obtained from the well-known least-squares equations R̂T (P ) #̂T (P ) = T Xt=1 XtXt 1; . . . ; T Xt=1 XtXt P!T : (2.3) The P P matrix R̂T is de ned as R̂T = T Xt=1 Xt iXt j : i; j = 1; . . . ; P! : Of course this least-squares estimator is closely related to the famous Yule-Walker estimator. More exactly the di erence of both quantities is of order OP (pmax(T )=T ) uniformly in P (T ) 2 [pmin(T ); pmax(T )] ; i.e. is asymptotically negligible. In principle we could use any other pT consistent procedure, but at several places it is quite convenient to work with this least-squares estimator because of its simple structure. Center the estimated innovations "̂t around zero by subtracting their sample mean and denote the empirical distribution of the centered innovations by F̂ c T : The usual residual bootstrap process X = (X t : t 2 Z ) is de ned as an autoregression of order P , parameter value #̂T (P ) and i.i.d. innovations (" t : t 2 Z ) distributed according to F̂ c T , i.e. X t = P X =1 â (P )X t + " t = 1 X =0 ̂ (P )" t ; t 2 Z : (2.4) 3 The moving average coe cients ̂ (P ), which are formally de ned as the power series coe cients of 1 PP =1 â (P )z 1 , can be computed recursively as follows (̂0(P ) = 1) ̂ (P ) = ^P X =1 â (P )̂ (P ) ; = 1; 2; . . . : As we will see in Section 4 it is not necessary to de ne the bootstrap innovations " t in such a complicated way. It su ces to ensure that they have on average the correct behaviour. Having the wild bootstrap idea in mind one could think of bootstrap innovations "yt = "̂t t ; t = 1; . . . ; T (2.5) for i.i.d. random variables ( t) with zero mean and second and third moment equal to one. Then the boostrap process is de ned as follows (Xy s = 0 for s < 0) Xy t = P X =1 â (P )Xy t + "yt = t 1 X =0 ̂ (P )"yt ; t = 1; . . . ; T : (2.6) So far both bootstrap proposals really generate processes in the bootstrap world. This is not the case for the following third proposal, namely applying the complete idea of wild bootstrap to autoregressive processes. On the basis of estimated innovations "̂t and bootstrap innovations "yt as above we de ne the wild bootstrap observations X+ t according to X+ t = P X =1 â (P )Xt + "yt ; t = 1; . . . ; T : (2.7) (2.7) is nothing else but a regression model with conditionally xed design. We will see in Section 4 that this proposal is usually not consistent. The wild bootstrap yields a consistent procedure only if the true underlying order of the autoregression coincides with P and if we are interested in the distribution of the parameter estimates. Even if we are interested in the distribution of the parameter estimate for an AR(1)t to data generated by an autoregression of order greater than one the wild bootstrap does not work. The reason for that is that the distribution which we intend to mimic by the bootstrap and even the asymptotic distribution really depends on the whole dependence structure of the process and this is not captured by the wild bootstrap. 3. Asymptotics for the usual Residual Bootstrap Let us begin the considerations in this section with a proof of the consistency of the usual residual bootstrap (cf. (2.4)) for the empirical autocovariance and autocorrelation. For this purpose we will frequently make use of the metric dp , which for two probability measures P and Q on IRk equipped with the Borel eld is de ned as dp(P;Q) = inf (EjjX Y jjp)1=p ; p > 0 ; (3.1) 4 where the in mum is over all IR2k random vectors (X;Y ) such that X P and Y Q : An excellent reference for properties of dp is Bickel and Freedman (1981). d2 is called Mallow's metric. A key result for the following is Proposition 3.1 For r = 2; 4 we have under the assumptions (A1), (A2) and (A3) together with E "2r 1 <1 dr F̂ c T ; F = oP (1) : We now can state the asymptotic validity of the usual residual bootstrap with respect to empirical autocovariances. Theorem 3.1 Assume that E "81 <1 and (A1), (A2) and (A3). Then we have d2 L pT ( ̂ T (h) (h))Kh=0 ;L pT ( ̂T (h) (h))Kh=0 = oP (1) : (3.2) ̂T ; denote the empirical and the theoretical autocovariance function of the underlying process, while ̂ T (h) = 1 T PT h t=1 X tX t+h ; (h) = E X tX t+h denote the corresponding quantities for the bootstrap process X : Moreover we have the following asymptotic normality of the bootstrapped autocovariances pT ̂ T (h) (h) : h = 0; . . . ;K )N (0; VK) in probability : (3.3) The asymptotic covariance matrix VK = " E"41 4 3 (i) (j) + 1 X k= 1 ( (k) (k i+ j) + (k + j) (k i))#Ki;j=0 is the same as for the empirical autocovariances (cf. Brockwell and Davis (1991), Proposition 7.3.2). To this theorem we have the following immediate corollary, which we state without proof, because the arguments for the application of the so-called -method (cf. Brockwell and Davis (1991), Proposition 6.4.3) are completely routine. Corollary 3.1 Under the same assumptions as in Theorem 3.1 we have the following asymptotic normality for the bootstrapped autocorrelations r̂ h = ̂ T (h)= ̂ T (0) ; h = 0; 1; 2; . . . : pT r̂ h r h : h = 1; 2; . . . ;K =)N (0;WK) in probability . (3.4) Notice that r (h) := (h)= (0) = r̂h + oP (T 1=2) , the empirical autocorrelation, for all h : The covariance matrix WK is of course equal to the asymptotic covariance of the usual empirical autocorrelations and can be found in Brockwell and Davis (1991), Theorem 7.2.1. 5 Remark. It is of some importance later on, that the covariance matrixWK of the empirical autocorrelations does not depend on the distribution of the innovations "t at all. This is in contrast to the asymptotic covariance matrix VK of the empirical autocovariances, which depend on the second and fourth moments of the innovations. We have seen in this section that the usual residual bootstrap for autoregressions with in nite order behaves as it should, at least with respect to smooth functions of the empirical autocovariances and the probably more relevant autocorrelations. We like to conclude this section with some remarks concerning parameter estimation. Suppose that we are interested in estimates of the parameters of an autoregressive t of given order q to the data. The data themselves still may arise from an in nite order underlying model. The theoretical parameters b of the tted model are given by the following smooth function of the autocorrelations 0B@ b1 ... bq 1CA = rji jj i; j = 1; . . . ; q 10B@ r1 ... rq 1CA : (3.5) If we consider plug-in estimates b̂ ; which are de ned as stated in (3.5) with rh replaced by the empirical autocorrelation r̂h ; then the usual residual bootstrap is consistent for such parameter estimates. Moreover, such a result holds for any smooth function of the autocovariance or the autocorrelation function. To be precise, we obtain for the parameter estimates of an AR(q)t as described above Corollary 3.2 Under the same assumptions as in Theorem 3.1 we have that the bootstrap distribution L pT h b̂ 1; . . . ; b̂ q b 1; . . . ; b q i X1 pmax(T ); . . . ;XT (3.6) converges weakly in probability to the same multivariate normal distribution as L pT b̂ b : = 1; . . . ; q . Here b̂ 1; . . . ; b̂ q T = r̂ ji jj i; j = 1; . . . ; q 10B@ r̂ 1 ...̂ r q 1CA ; whereas (b 1; . . . ; b q) are similarily de ned (replace r̂ h by r h , cf. Corollary 3.1). We may replace (b 1; . . . ; b q) by (b̂1; . . . ; b̂q) , since the di erence of both quantities is of smaller order than T 1=2 : Proof: This result is a direct consequence of Corollary 3.1 and the so-called method (cf. Brockwell and Davis (1991), Proposition 6.4.3). In Section 6 of the paper we will discuss in more detail a muchmore complicated functional of the whole autocorrelation function. But before let us focus our attention to a wild bootstrap proposal for autoregressions. 6 4. Asymptotics for the Wild Bootstrap The principal idea of a wild bootstrap procedure has been stated already in Section 2 of the paper. Let us begin this section with a negative result, namely that the wild bootstrap proposal usually leads to an inconsistent procedure. Since the wild bootstrap observations X+ t are derived from a regression model it makes no sense to consider autocorrelation structure. That is why we focus on parameter estimates, only. Assume that we intend to approximate the distribution of the parameter estimates of an AR(q)t as was described at the end of the previous section. A tted model of order q in the wild bootstrap world is de ned by the following theoretical parameter values which we intend to estimate in the bootstrap world. argminb1 ;...;bqE+ T Xt=1 X+ t q X =1 b Xt !2 = 1 T T Xt=1 Xt Xt : ; = 1; . . . ; q! 1 1 T T Xt=1 E+X+ t Xt : = 1; . . . ; q!T (4.1) =: b̂1; . . . ; b̂q T : Here E+X+ t = PP =1 â (P )Xt : Since the â (P ) are usual least squares estimators, cf. (2.3), we obtain by simple algebra that the b̂ ; as de ned above, coincide with the least-squares estimators of order q, i.e. b̂1; . . . ; b̂q T = T Xt=1 Xt Xt ! 1 ; =1;...;q T Xt=1 XtXt ! =1;...;q ; whenever P (T ) q : Let us consider estimators for the b̂ in the wild bootstrap world which are de ned as follows b̂+1 ; . . . ; b̂+q T := T Xt=1 Xt Xt ! 1 ; =1;...;q T Xt=1 X+ t Xt ! =1;...;q : The following proposition gives us the asymptotic distribution of these quantities. Proposition 4.2 Assume E"41 <1 and (A1)-(A3). Then we have in probability L pT h b̂+1 ; . . . ; b̂+q b̂1; . . . ; b̂q i X1; . . . ;XT =) N 0; 2 (q) 1 ; (4.2) where (q) = (EX1 X1 ) ; =1;...;q : Proof: Let us denote ̂T (q) = 1 T PtXt Xt : ; = 1; . . . ; q : Then we have from (4.1) pT h b̂+1 ; . . . ; b̂+q b̂1; . . . ; b̂q i = ̂ 1 T (q) 1 pT Xt X+ t E+X+ t Xt ! =1;...;q : 7 A usual CLT for triangular arrays of independent random variables (cf. G anssler and Stute (1977), Korollar 9.2.9) now yields the desired result, since we can show that E+ 1 pT Xt X+ t E+X+ t Xt ! 1 pT Xt X+ t E+X+ t Xt ! = 1 T Xt E+ X+ t E+X+ t 2Xt Xt = 1 T Xt "̂2tXt Xt ; cf. (2.5) !T!1 2 EX1 X1 : The last convergence follows by direct computation from Lemma 8.2, 8.3 and the ergodic theorem. Remark. The asymptotic variance given in Proposition 4.2 is equal to the asymptotic variance of the Yule-Walker or the least-squares parameter estimates in AR(q)-models, but it is not equal to the asymptotic variance of Yule-Walker parameter estimates of an AR(q)t to autoregression of higher order. Even in the case where the underlying autoregression is of nite but higher order than q the wild bootstrap as introduced above is not consistent. The positive message is, that if we intend to t an autoregression of order q to an underlying process of exactly this order, then the wild bootstrap is consistent. In this speci c situation the wild bootstrap has the advantage over the usual bootstrap, that it even works for models with conditional heteroskedasticity. Let us state this assertion as a seperate result. Theorem 4.2 Assume that the underlying autoregression is of order q 2 IN , (A1)-(A3), E"21 = 1 ; E"41 <1 and that we have heteroskedasticity, e.g. Xt = q X =1 b Xt + (Xt 1)"t ; t 2 Z : (4.3) : IR! (0;1) denotes a bounded function, which is also bounded away from zero. Then we have in probabilityL pT h b̂+1 ; . . . ; b̂+q b̂1; . . . ; b̂q i ) N 0; (q) 1 E 2(X0)X1 X1 ; =1;...q (q) 1 : Since the asymptotic variance is equal to the asymptotic variance of the least-square parameter estimates, this means that the wild bootstrap proposal for AR(q)-parameter estimation is consistent for heteroskedastic autoregression of order q . Proof: The arguments are quite similar to the ones given in the proof of Proposition 4.2, since 1 T Xt "̂2tXt Xt = 1 T Xt "2t 2(Xt 1)Xt Xt + oP (1)!T!1 E 2(X0)X1 X1 ; 8 in probability. To see the rst equality we make use of (8.8) and the fact that the fourth moment of Xt is bounded. The last convergence is a usual ergodic property. Remark. The assumptions on the function in Theorem 4.2 ensure strict stationarity and ergodicity of the model (cf. Masry and Tj stheim (1995), Lemma 3.1). Without any doubt these assumptions can be relaxed. The asymptotic normality of the least-squares estimator in model (4.3) can be obtained from a usual CLT for martingale di erence schemes (cf. G anssler and Stute (1977), Satz 9.2.3) and the ergodic theorem. If we want to modify the wild bootstrap in order to obtain a consistent procedure for more general situations we may proceed as follows. Let "yt be de ned as in Section 2, cf. (2.5). Suppose now that bootstrap observations Xy t are given as Xy t = P X =1 â (P )Xy t + "yt : (4.4) This bootstrap proposal preserves the whole dependence structure of the process. The bootstrap observations are no longer independent; for that reason one may hesitate to call the proposal a wild bootstrap procedure. Under suitable assumptions we obtain that this proposal is consistent for empirical autocorrelations (not for empirical autocovariances, which is perhaps of minor importance). This result ensures that the modi ed wild bootstrap proposal is an appropriate tool for approximating the distribution of a tted autoregression of xed order, q say, to the data even if the underlying model has a di erent autoregressive order, including in nity. In the following we state rigorous results, which make these remarks precise. Let us start with an investigation of the empirical autocovariances for the modi ed wild bootstrap. As mentioned above, we do not obtain consistency for this bootstrap procedure. The main reason for this is that the asymptotic distribution of the empirical autocovariances depends on fourth moments of the innovations, which are not mimicked correctly by the construction of "yt : Nevertheless we obtain as a corollary (stated as Theorem 4.3) the consistency of the modi ed wild bootstrap for empirical autocorrelations. Proposition 4.3 Assume (A1)-(A3) and E"81 <1 : Then we have the following asymptotical distribution for the empirical autocovariances of the modi ed wild bootstrap process pT ̂y T (h) y(h) : h = 0; 1; 2; . . . ;K ) N 0; V 0 K in probability, (4.5) where (recall the de nition of Xy t , cf. (2.6)) y(h) = 1 T T h Xt=1 EyXy tXy t+h = 1 T T h Xt=1 t 1 X =0 ̂ (P )̂ +h(P )"̂2t : (4.6) The covariance matrix V 0 K is given by the following entries (r; s = 0; . . . ;K) " (E 4 1 1)E"41 4 2 (r) (s) + 1 X j= 1 ( (j) (j r + s) + (j + r) (j s))# (4.7) 9 Remark. V 0 K di ers from the asymptotical covariance of the empirical autocovariances (cf. Theorem 3.1) by the factor (E 4 1 1)E"41= 4 2 instead of E"41= 4 3 : Since E 2 1 = 1 we have E 4 1 1 : Moreover, if we realize the distribution of 1 by the unique two-point distribution with E 1 = 0 and E 2 1 = E 3 1 = 1 ; then we obtain E 4 1 = 2 : In any case the modi ed wild bootstrap procedure is not consistent concerning autocovariances. But this is not a big problem, since for the much more relevant autocorrelations we obtain consistency for the modi ed wild bootstrap. This is the contents of the following theorem. Theorem 4.3 Under the same assumptions as in Proposition 4.3 we have the following asymptotical distribution for empirical autocorrelations r̂y h = ̂y T (h)= ̂y T (0) of the modi ed wild bootstrap process (abbreviate y(h)= y(0) by ry h) : pT r̂y h ry h : h = 1; 2; . . . ;K =)N (0;WK) in probability. (4.8) WK is de ned in Corollary 3.1. Proof: We obtain the assertion from Brockwell and Davis (1991), Proposition 6.4.3, since we have for the map g : IRK+1 ! IRK; (x0; x1; . . . ; xK) ! (x1=x0; . . . ; xK=x0) that r̂y 1; . . . ; r̂y K = g ̂y T (0); . . . ; ̂y T (K) : Exactly along the lines of Section 3 (cf. Corollary 3.2) we easily obtain that the modi ed wild bootstrap is an appropriate tool for approximating the distribution of an tted autoregression of xed order, q say, to the data. More exactly we have the following corollary to Theorem 4.3. Corollary 4.3 Under the same assumptions as in Proposition 4.3 we have that the modi ed wild bootstrap distribution L pT h b̂y1; . . . ; b̂yq by1; . . . ; byq i X1 pmax(T ); . . . ;XT (4.9) converges weakly in probability to the same multivariate normal distribution as L pT b̂ b : = 1; . . . ; q . Here b̂y1; . . . ; b̂yq T = r̂y ji jj i; j = 1; . . . ; q 10B@ r̂y 1 ...̂ ry q 1CA and (by1; . . . ; byq)T is analogously de ned by replacing r̂y h by ry h (cf. Theorem 4.3 for a de nition of ry h). 10 5. Accuracy of the Bootstrap Proposals In a remarkable paper by Janas (Janas (1993 a,b)) we nd conditions under which an Edgeworth expansion is valid for both the statistics of interest and their bootstrap counterparts. But note that Janas considered the so-called periodogram-based bootstrap procedure, which is quite di erent from the bootstrap proposals de ned herein. Nevertheless, under the assumptions made in the paper by Janas, we obtain that we can approximate the following probability P npT ( ̂T (0) (0); . . . ; ̂T (K) (K)) 2 Co (5.1) uniformly over all measurable and convex subsets C of IRK+1 by an Edgeworth expansion T;3(C) up to an error term of order o(T 1=2) : Since autocorrelations are smooth functions of the sample autocovariances, e.g. (r̂1; . . . ; r̂K) = (g1( ̂T (0); . . . ; ̂T (K)); . . . ; gK( ̂T (0); . . . ; ̂T (K))) ; we obtain for standardized expressions like pTV 1=2 T;K (r̂1 r1; . . . ; r̂K rK) =: ST ; (5.2) where VT;K denotes theK K covariance matrix ofpT (r̂1 r1; :::; r̂K rK) , the following Edgeworth approximation. sup P fST 2 Cg ZC 1 + T 1=2p3(x) d (x) = o(T 1=2) ; (5.3) where the coe cients of the polynomial p3 depend continuously on the cumulants of order less than or equal to 3 of the following Taylor approximation (abbreviate GT = pT ( ̂T (0) (0); . . . ; ̂T (K) (K)) W (i) T;3 = (5gi)( (0);...; (K))GTT + T 1=2 2! GT (Dgi)( (0);...; (K))GTT : The supremum in (5.3) is over all measurable and convex sets C in IRK ; cf. Janas (1993a), Lemma 5.5 or Bhattacharya and Denker (1990), p. 22-32. One main point in Janas (1993) was that the cumulants of W i T;3 of order 3 or less do not depend on cumulants or moments of the innovations. Since our bootstrap proposals (usual bootstrap and modi ed wild bootstrap) are based on quite analogous autoregressive processes, we can compute formally similar edegworth expansions of rst order for these two bootstrap proposals. We obtain that the cumulants of order 3 or less of the corresponding Taylor approximation coincide (up to a su ciently large order) with the cumulants of the non-bootstrapped quantities. This indicates that both bootstrap proposals catch the rst higher order terms, which means that both bootstrap proposals outperform the normal approximation. Since parameters of tted autoregressive models of xed order are smooth functions of the autocorrelation, a similar remark holds true for usual or modi ed wild bootstrap approximations of these statistics. It should be stressed here that we do not have a rigorous proof of the validity of an Edgeworth expansion for our bootstrap proposals and that the results of Janas (1993) for bootstrap quantities do not cover any of our proposals. 11 6. Bootstrapping Spectral Density Estimators In Section 5 we already mentioned the papers Janas (1993 a,b). Janas proposed a periodogram-based bootstrap procedure which is able to mimic the behaviour of statistical quantities which can be written as functionals of the periodogram. In this context we also refer to a paper by Franke and Hardle (1992) where the same bootstrap proposal as in Janas is applied to smoothed periodogram estimators of the spectral density. Let us discuss in this section how the bootstrap proposals of Section 3 and 4 apply to spectral density estimation. Most popular estimates of the spectral density are obtained from smoothing the so-called periodogram. As is argued for example in Brockwell and Davis (1991), Section 10.4, smoothed periodogram estimators are quite similar to so-called lag window estimators de ned as follows '̂T ( ) = 1 2 X jhj M w h M ̂T (h)e ih ; 0 ; (6.1) where w : [ 1; 1] ! [0;1) denotes a continuous window function, which is assumed to satisfy (W) 0 w(u) w(0) = 1 , w(u) = w( u) for all juj 1 and limu!0 1 w(u) jujq > 0 for some q > 0 Asymptotic normality of '̂T is given in the monograph of Anderson (1971), Theorem 9.4.1. The result is as follows. Proposition 6.4 Assume (A1), (A2), E"41 < 1 , Ph h2j (h)j < 1 and that w : [ 1; 1] ! [0;1) is continuous and satis es (W). Then we have for M(T ) ! 1 but M(T )=T ! 0 and T=M(T )5 ! 0 as T !1 s T M(T ) ('̂T ( ) '( ))) N (0; 2( )) ; (6.2) where 2( ) = ( 2'2( ) R 1 1 w2(u) du if = 0; '2( ) R 1 1w2(u) du if 0 < < : (6.3) ' : [0; ] ! IR denotes the underlying spectral density of the process, which has the following representation '( ) = E"21 2 1 1 X =1 a e i 2 = 1 2 1 X h= 1 (h)e ih : (6.4) Proof: Cf. Anderson (1971), Theorem 9.4.1. The representation (6.4) of the spectral density ' for AR(1)-processes can be found in Brockwell and Davis (1991), Corollary 4.3.2 and Theorem 4.4.1. 12 We have the following representation of '̂T ( ) '( ) (cf. (6.1) and (6.4)) r T M ('̂T ( ) '( )) = r T M 1 2 X jhj M w( h M ) f ̂T (h) (h)g e ih r T M 1 2 X jhj M 1 w( h M ) (h)e ih r T M 1 2 X jhj>M (h)e ih : From (W) and Ph h2j (h)j < 1 we obtain that the second and third summand in the above representation converge to zero, i.e. the asymptotic distribution is completely determined by ST ( ) =r T M 1 2 X jhj M w( h M ) f ̂T (h) (h)g e ih : (6.5) This strongly suggests to use S T ( ) =r T M 1 2 X jhj M w( h M ) f ̂ T (h) (h)g e ih (6.6) as a bootstrap approximation os ST : Here we make use of the usual residual bootstrap introduced in Section 2 and 3. We have the following result Theorem 6.4 Assume (A1)-(A3) and E"41 < 1 : Then we have in probability for all 0 d2 (L (S T ( )) ;L (ST ( )))! 0 as T !1 : d2 is de ned in (3.1). Under the assumptions of Proposition 6.4 we have the same asymptotic normal distribution for S T ( ) as for the lag-window spectral density estimator (6.1). 7. Simulations In this section we try to demonstrate the nite sample properties of the usual residual and the modi ed wild bootstrap. For this purpose let us consider the following three models Model I (AR(1)) : Xt = 0:9 Xt 1 + "t Model II (ARMA(2,1)) : Xt = 1:29 Xt 1 + 0:83 Xt 2 + "t + 0:5 "t 1 Model III (ARMA(2,2)) : Xt = 0:1 Xt 1 + 0:8 Xt 2 + "t + 0:1 "t 1 + 0:8 "t 2 In model I we deal with estimators of an AR(1)t, i.e. with estimators of the underlying coe cient b1 = 0:9 and the usual residual bootstrap, cf. Section 3. All procedures are based on a sample size of T = 100 of the underlying process. The innovations ("t) are 13 assumed to be double-exponentially distributed. In Figure 1a we compare the simulated density of the distribution pT (b̂1 b1) (thick curve), cf. (3.5), with three independent usual residual bootstrap approximations (thin curves) based on di erent and randomly chosen samples of size 100 of the underlying model. Insert Figure 1a around here These bootstrap approximations have been computed using the order P (T ) = 1 for the bootstrap autoregressive process. Figure 1b contains similar densities, but now we use P (T ) = 3 ; i.e. we use an over tted AR(3)-model in the bootstrap world. But again we use the usual residual bootstrap procedure and the parameter estimator of an autoregressive t of order 1. Insert Figure 1b around here Now let us turn to an underlying autoregression of in nite order. i.e. to model II. Figures 2a and 2b demonstrate the behaviour of the modi ed wild bootstrap, cf. Section 4. All simulated densities correspond to the distribution of estimators of the rst autocorrelation r1 = 0:626 : Insert Figures 2a and 2b around here The thick curves in Figure 2a and 2b represent the density of the distribution of pT (r̂1 r1) ; i.e. the distribution of the sample autocorrelation (sample size T = 200) : The thin curves are independent modi ed wild bootstrap approximations. In Figure 2a we use an AR(2)-model in the bootstrap world, i.e. P (T ) = 2 ; while Figure 2b contains similar results for P (T ) = 5 : From the simulations one gets the impression that the selection of the order P (T ) ; i.e. the order of the autoregressive model in the bootstrap world, is not a very crucial parameter. Moreover, over tting seems to be not very problematic. Finally, we consider lag-window estimators of the spectral density in model III, cf. Section 6. As a window function we use the so-called triangular window, i.e. w(u) = (1 juj)[ 1;1](u) : For the sample size T = 200 a truncation atM = 10 seems to be appropriate. We use the usual residual bootstrap procedure to produce a pointwise con dence band at level 0.95 for the underlying spectral density (cf. Figure 3a). The dashed line refers to the underlying spectral density, while the solid lines refer to the pointwise con dence band. The underlying bootstrap autoregressive process is of order P (T ) = 4 : In comparison, we report in Figure 3b a classical pointwise con dence band (level 0.95) for the spectral density based on a Chi-square approximation (cf. Brockwell and Davis (1991), Chapter 10.5). 14 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.0 0.2 0.4 0.6 0.8 1.0 ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ..... ....... .................... ...... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ...................................................................................................................................................... ................................................................................................................................................................................... ................................................................................................................................................................................................................... ..... ............................................................................................................ Figure 3a 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.0 0.2 0.4 0.6 0.8 1.0 ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ..... ....... .................... ...... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ...................................................................................................................................................................................................................................................................................................... .................................................................................................................................... ................................................................................................................................................................................... ......................................................................................................................................................... Figure 3b Remark. All simulated densities are computed on the basis of 1000 Monte Carlo replications. For the smoothing we have used a usual Nadaraya-Watson kernel density estimator with kernel K(u) = 3=4(1 u2)1[ 1;1](u) and bandwidth h > 0 : 8. Auxiliary Results and Proofs At several places we heavily make use of the following inequalities, which mainly concern the behaviour of the moving average coe cients ̂ (P ) of the tted autoregression of order P to the given data set. Let us consider the following two power series expansions. Both series converge at least for all complex z with magnitude less than or equal to one. A(z) = 1 1 X =1 a z and Ap(z) = 1 p X =1 a (p)z : (8.1) Here (a : 2 IN) denote the coe cients of the underlying autoregression (1.1), whereas (a (p) : = 1; . . . ; p) denote the coe cients of a (theoretically) best AR(p)t in L2 distance, i.e. the coe cients (a1(p); . . . ; ap(p)) T = #(p) are de ned uniquely as the argmin of E (Xt Pp =1 c Xt )2 , which is equivalent to #(p) = R(p) 1(r1; . . . ; rp)T : (8.2) 15 From assumption (A2) we obtain the following moving average representation of the underlying process X : Xt = 1 X =0 "t ; t 2 Z ; (8.3) where ( : = 0; 1; 2; . . .) denote the power series coe cients of (1 P a z ) 1 : According to Theorem 2.2 of Baxter (1962) we have a constant C > 0 and po 2 IN such that for all integers p po p X =1 ja (p) a j C 1 X =p ja j : (8.4) This inequality easily implies that for all large p the polynomial Ap , cf. (8.1), has no zeroes with magnitude less than or equal to one. Moreover we have the following result. Lemma 8.1 There exist > 0 and po 2 IN , such that for all p po inf jzj 1+1=p jAp(z)j : Proof: Suppose that the assertion of Lemma 8.1 is false. Then there exists a sequence p(k) of integers converging to in nity as k ! 1 and a sequence of complex numbers zk with jzkj 1 + 1=p(k) such that Ap(k)(zk)! 0 as k !1 : Without loss of generality we assume that zk ! zo : Necessarily jzoj 1 ; more exactly jzoj = 1 : We will show that A(zo) = 0 ; which is a contradiction to our assumptions. To this end observe A(zo) = 1 1 X =1 a z o = Ap(k)(zk) + p(k) X =1 a (z k z o ) + p(k) X =1 (a (p(k)) a ) z k + 1 X =p(k)+1 a z o = o(1) ; because of our assumptions on zk ; (A2), (8.4) and the Theorem of dominated convergence. Note that jzkj jzkjp(k) (1 + 1=p(k))p(k) 3 for all p(k) : This implies A(zo) = 0 : The coe cients of the moving average representation (8.3) can formally be de ned as follows 1 + 1 X =1 z = 1 1 X =1 a z ! 1 ; jzj 1 : (8.5) Because of Lemma 8.1 we may de ne for all p large enough 1 + 1 X =1 (p)z = 1 p X =1 a (p)z ! 1 ; jzj 1 + 1=p : (8.6) We obtain the following result. 16 Lemma 8.2 There exists a constant C > 0 such that for all large p 1 X =0 j (p) j C 1 X =p ja j : (8.7) Obviously, the right hand side converges to zero as p!1 . Proof: Since a = (1; a1; a2; . . .) 2 `1 = n(z : 2 IN o) C P jz j <1o and 1 P a z 6= 0 for all jzj = 1 , a has a multiplicative inverse a 1 = (1; 1; 2; . . .) 2 `1 , i.e. a a 1 = 1; n n 1 Xk=0 an k k : n 2 IN! = (1; 0; 0; . . .) ; where a b is the convolution of a and b : Cf. Zelazko (1973), Theorem 8.11 and Assertion 9.4. The coe cients are the power series coe cients of (1 P a z ) 1 : Because of a 1 2 `1 we have P j j <1 . Furthermore, for all large p ; (1; a1(p); . . . ; ap(p); 0; 0; . . .) a(p) 2 `1 and is invertible with inverse a(p) 1 = (1; 1; 2; . . .) , cf. (8.6). Finally observe that 1 X =0 j (p) j = a(p) 1 a 1 = a(p) 1(a a(p))a 1 a(p) 1 a 1 ja a(p)j a 1 + a 1 2 ja a(p)j ; which leads to a(p) 1 a 1 ja 1j2 ja a(p)j 1 ja 1j ja a(p)j : ja a(p)j =Pp =1 ja a (p)j +P1 =p+1 ja j together with (8.4) implies the assertion of Lemma 8.2 . Remark. If a = 0 for all po , i.e. X is a nite order autoregression, then P1 =0 j (p) j = 0 for all p po : Finally we need a bound for ĵ (p) (p)j : Here we denote by ̂ (p) the coe cients of the series (1 Pp =1 â (p)z ) 1 : Recall that #̂T (p) = (â1(p); . . . ; âp(p))T denotes the least squares parameter estimator, cf. (2.2). From Hannan and Kavalieris (1986), Theorem 2.1, we obtain under our assumptions max 1 p jâ (p) a (p)j = OP r log T T ! (8.8) uniformly in p pT , p2T = o(T= log T ) : This yields, again uniformly in p pT ; p X =1 jâ (p) a (p)j 1 + 1 p = OP pr log T T ! = oP (1) : (8.9) 17 Because of (8.9) and Lemma 8.1 we have with large probability that the polynomial 1 Pp =1 â (p)z also has no zeroes with magnituse less than or equal to 1 + 1=p : Because of this we may apply Cauchy's inequality for holomorphic functions in order to obtain the following result. Lemma 8.3 We have uniformly in p pT , p2T = o(T= log T ) , and uniformly in 2 IN ĵ (p) (p)j p 1 + 1 p OP r log T T ! : (8.10) Proof: Cauchy's inequality (cf. Ahlfors (1966) or Rudin (1987), Theorem 10.26) together with (8.8), (8.9) and Lemma 8.1 yields ĵ (p) (p)j 1 1 + 1 p max jzj=1+1=p 1 X a (p)z ! 1 1 X â (p)z ! 1 = p 1 + 1 p OP r log T T ! ; which is the assertion. Proof of Proposition 3.1: Let us denote by F̂T the empirical distribution of the unobservable innovations "1; . . . ; "T . Because of Lemma 8.4 in Bickel and Freedman (1981) it su ces to show that dr F̂ c T ; F̂T converges to zero in probability. To see this let J be Laplace distributed on f1; . . . ; Tg and de ne random variables Y1; Y2 with marginals F̂T , F̂ c T , respectively, as follows Y1 = "J ; Y2 = "̂J 1 T T Xt=1 "̂t : For the case r = 2 we obtain d22 F̂ c T ; F̂T E (Y1 Y2)2 = 1 T Xj "̂j "j 1 T Xt "̂t!2 6 T Xj ("̂j "j)2 + 3 1 T Xt "t!2 | {z } oP (1) = 6 T Xj P X =1 (â (P ) a )Xj 1 X =P+1 a Xj !2 + oP (1) : 18 The proof can be concluded from the following three results. E 1 X =P ja jjXj j!2 E X2 1 1 X =pmin ja j!2 = o(1) ; since pmin !T!1 1 : 1 T Xj P X =1 (â (P ) a (P ))Xj !2 max 1 p pmax(T ) max 1 p jâ (p) a (p)j2 1 T Xj pmax X =1 jXj j!2 = OP p2max(T ) log T T = oP (1) ; because of (8.8). 1 T Xj P X =1 (a (P ) a )Xj !2 = p X ; =1 (a (P ) a ) (a (P ) a ) 1 T Xj Xj Xj pmax X p=pmin p X ; =1 ja (p) a j ja (p) a j 1 T Xj Xj Xj ( ) + p X ; =1 ja (P ) a j ja (P ) a j j ( )j ( ) = OP (T 1=2) pmax X p=pmin p X =1 ja (p) a j!2 + EX2 1 P X =1 ja (P ) a j!2 OP (T 1=2) pmax(T ) + EX2 1 1 X =pmin ja j!2 ; cf. (8.4) = oP (1) ; by assumption (A2) and (A3). To see equality ( ) observe that E 1 pT PjfXj Xj ( )g 2 is uniformly bounded in and and that j ( )j EX2 1 : Proof of Theorem 3.1: For reasons of not too complicated notation, we restrict our attention to the case K=0, only. Denote by ((" i ; "i) : i 2 Z ) a sequence of i.i.d. bivariate random vectors with arbitrary dependence structure between " 1 and "1 up to the given marginals " 1 F̂ c T and "1 F : The square of (3.2) is bounded through 1 T inf E T Xt=1 X t 2 E X t 2 X2 t EX2 t !2 19 1 T E0@ T Xt=1 24 1 X =0 ̂ (P )" t ! E " 12X ̂2 (P ) X "t !2 + E"21X 2 351A2 : It su ces to consider the following two expectations. E0@Xt 24 X " t !2 X "t !2 X 2 E" 12 E"21 351A2 (8.11) and E0@Xt 24 X ̂ (P )" t !2 X " t !2 X ̂2 (P ) 2 E" 12351A2 2 E0@Xt 24 X (̂ (P ) ) " t !2 X (̂ (P ) )2 E" 12351A2 (8.12) +8 E Xt "X (̂ (P ) ) " t X " t X (̂ (P ) ) E" 12#!2 : Compute the rst expectation on the right hand side of (8.12) and obtain the following bound T Xt=1 t Xs=1 X (̂ +t s(P ) +t s)2 (̂ (P ) )2 E" 14 3 E" 12 2 + T Xt=1 t Xs=1 X ; ̂ +t s(P ) +t s ̂ +t s(P ) +t s ̂ (P ) ̂ (P ) E" 12 2 Use the fact that E" 14 and E" 12 are bounded in probability because of Proposition 3.1 in order to obtain with the help of Lemma 8.2 and 8.3 that the last expression can be bounded through OP 0@sp7=2 max(T ) T log T + 1 X =pmin(T ) ja j1A = oP (1) : Exactly along the same lines we obtain a similar bound for the second expectation of the right hand side of (8.12). This bound also vanishes in probability. It remains to consider (8.11). A tedious but direct computation gives us the following bound for this expectation OP pE(" 1 "1)2 + pE(" 1 "1)4 = OP d2(F̂ c T ; F ) + d24(F̂ c T ; F ) : Proposition 3.1 completes the argument. 20 Proof of Proposition 4.3 : For the sake of simplicity we restrict our attention again to the case K=0. Recall from (2.6) the de nition of Xy t and obtain pT ̂y T (0) y(0) = 1 pT T Xt=1 Xy t 2 y(0) = 1 pT Xt 24 t 1 X =0 ̂ (P )"yt !2 t 1 X =0 ̂2 (P ) "̂2t 35 : For xed m 2 IN abbreviate Um t;T := m X =0 ̂ (P )"yt !2 m X =0 ̂2 (P )"̂2t (8.13) and observe that these are centered and m-dependent random variables. We may conclude the proof of Proposition 4.3 from Brockwell and Davis (1991), Proposition 6.3.9, if we can verify (8.14) (8.16). 1 pT T Xt=1 Um t;T =)N (0; 2 m) in probability ; (8.14) where 2 m = ((E 4 1 1)E"41 2 4) (Pm =0 2 )2+2 4 (Pm =0 2 )2 + 2Pmh=1 Pm h =0 +h 2 : 2 m !m!1 2 = = ((E 4 1 1)E"41 2 4) 1 X =0 2 !2 + 2 40@ m X =0 2 !2 + 2 1 Xh=1 1 X =0 +h!21A (8.15)= (E 4 1 1)E"41 2 4 4 (0)2 + 2 (0)2 + 2 1 Xh=1 (h)2! and lim m!1 lim sup T!1 Ey 1 pT T Xt=1 U t 1 t;T Um t;T !2 = 0 in probability. (8.16) To see (8.13) we use the following Lemma 8.4. We have EyUm t+h;TUm t;T = m X ; =0 m X ; =0 ̂ (P )̂ (P )̂ (P )̂ (P ) Ey "yt+h "yt+h "yt "yt m X ; =0 ̂2 (P )̂2 (P )"̂2t+h "̂2t = m h X =0 ̂2 +h(P )̂2 (P ) hEy("yt )4 3 "̂4t i + 2 m h X ; =0 ̂ +h(P )̂ +h(P )̂ (P )̂ (P )"̂2t "̂2t : 21 Observe that by de nition of "yt ; cf. (2.5), we have Ey("yt)4 = E 4 1 "̂4t : From Proposition 3.1 and Lemma 8.3 of Bickel and Freedman (1981) we have for all ; = 0; . . . ;m : (i) 1 rT rT X =1 "̂4t !T!1 E"41 in probability (ii) 1 rT rT X =1 "̂2t "̂2t ! (E"21)2 = 4 ; 6= E"41 ; = in probability. frTg denotes a sequence of integers converging to in nity and t1 < t2 < . . . < trT T for all T 2 IN : We nally obtain from (i), (ii) and Lemma 8.2 and 8.3 for every xed m 2 IN 1 pT rT X =1 EyUm t +h;TUm t ;T !T!1 c(h) in probability , where c(h) = (E 4 1 1)E"41 2 4 m h X =0 2 2 +h + 2 4 m h X =0 +h!2 : Simple algebra yields 2 m = c(0) + 2 m Xh=1 c(h) = (E 4 1 1)E"41 2 4 m X =0 2 !2 + 2 40@ m X =0 2 !2 + 2 m Xh=1 m h X =0 +h!21A > 0 : The following Lemma 8.4 now ensures (8.13). (8.15) is obvious. Finally we conclude (8.16) as follows. Ey 1 pT T Xt=1 U t 1 t;T Um t;T !2 = 1 T T X s;t=1X >m ̂2 +t s(P )̂2 (P )nEy("ys )4 3 "̂4s o + 2 T T X s;t=1 X _ >m ̂ +t s(P )̂ (P )̂ +t s(P )̂ (P )"̂2s "̂2s : Repeated application of Lemma 8.2 and 8.3 yields that lim sup T!1 E 1 pT T Xt=1 U t 1 t;T Um t;T !2 = O 1 X =m 2 !!m!1 0 by our assumptions. 22 In the above proof we make use of a version of the CLT for m-dependent random variables. This version is a slight modi cation of results known in the literature. This is especially true for a CLT for strictly stationary m-dependent sequences, cf. Brockwell and Davis (1991) Theorem 6.4.2. The following version allows for triangular arrays which are on average stationary, only (see Lemma 8.4 for precise de nition). Lemma 8.4 CLT for m-dependent sequences Assume that for each T 2 IN real-valued, centered and m-dependent random variables fUt;T : t = 1; . . . ; Tg are given. Further assume For h 2 IN o ; each sequence frt : t 2 INg of positive integers converging to in nity and arbitrary t1; t2; . . . with trT T h : 1 rT rT X =1 E (Ut +h;TUt ;T )!T!1 c(h) ; h 2 INo ; (8.17) where the function c full lls c(0) + 2 m Xk=1 c(h) = 2 > 0 : 1 T 1+ T Xt=1 E jUt;T j2(1+ ) !T!1 0 for some > 0 : (8.18) Then we have lim T!1Var 1 pT T Xt=1 Ut;T! = 2 (8.19) and 1 pT T Xt=1 Ut;T =)N (0; 2) : (8.20) Remark. The convergence part in assumption (8.17) of Lemma 8.4 is for example implied by the stricter assumption EUtT+h;TUtT ;T ! c(h) as T !1 for every sequence ftT : T 2 INg with tT 2 f1; . . . ; Tg and tT !1 as T !1 : Because of this, Lemma 8.4 generalizes Theorem 6.4.2 of Brockwell and Davis (1991) slightly. Proof: To see (8.19) compute for T > m Var 1 pT T Xt=1 Ut;T! = 1 T T Xt=1 T Xs=1 E(Us;TUt;T ) = = 1 T T 1 X t= (T 1) T jtj Xs=1 E Us+jtj;TUs;T = m X t= m 1 T T jtj Xs=1 E Us+jtj;TUs;T ; m-dependence !T!1 m X t= m c(jtj) = c(0) + 2 m Xt=1 c(t) > 0 ; because of (8.16) : 23 To prove (8.20) consider for k > 2m and r rT = [T=k] YT;k := 1 pT r Xj=1 U(j 1)k+1;T + + Ujk m;T | {z } =Zj;T : The triangular array fZt;T : t = 1; . . . ; rgT2IN , which consists of independent random variables in each row, full lls (for the sake of simplicity we assume that T = kr m for some integer k) 1 T E T Xt=1 Ut;T r Xj=1 Zj;T!2 = 1 T r 1 Xj=1 E (Ujk m+1;T + + Ujk;T )2 = r 1 T m X =1 m X =1 1 r 1 r 1 Xj=1 E (Ujk m+ ;TUjk m+ ;T ) | {z } !c(j j) as T!1 because of (8.17). !T!1 1 k X jhj m(m jhj) c(jhj) : Because of this we obtain for each " > 0 lim k!1 lim sup T!1 P ( 1 pT T Xt=1 Ut;T YT;k > ") = 0 : According to Proposition 6.3.9, Brockwell and Davis (1991), it su ces to deal with YT;k : We have in mind to apply the CLT for triangular arrays. To this end compute 1 T r Xj=1 EZ2 j;T = 1 T r Xj=1 E U(j 1)k+1;T + + Ujk m;T 2 = 1 T r Xj=1 k m 1 X t= (k m 1) k m jtj Xs=1 E U(j 1)k+s+jtj;TU(j 1)k+s;T !T!1 X jhj m k m jhj k c(jhj) because of (8.17) . Since a Ljapunov-condition is full lled, i.e. for some > 0 1 T 1+ r Xj=1 E jZj;T j2(1+ ) 1 T 1+ Xj E k m Xs=1 U(j 1)k+s;T !2(1+ ) k1+2 T 1+ Xj Xs E U(j 1)k+s;T 2(1+ ) k1+2 1 T 1+ T Xt=1 E jUt;T j2(1+ ) !T!1 0 because of (8.18), 24 we obtain eventually from the CLT for triangular arrays (see for example Ganssler andStute (1977), Korollar 9.2.9)YT;k = 1pT rXj=1 Zj;T =) N 0@0; Xjhj m 1 m+ jhjkc(jhj)1A :Observe thatPjhj m 1 m+jhjk c(h) !k!1Pjhj m c(h) = 2 > 0 to conclude theproof.Proof of Theorem 6.4 : We have (let r;s = 1 if r = s and 0 otherwise)(h) =E"21 1X; =0+h; and (h) = E X tX t+h = E ("1)2 1X; =0 ̂ (P )̂ (P ) +h; :These expressions are a consequence of the moving average representation of X , cf. (8.3),and of X ; cf. (2.4). From the de nition of ̂T (h) and ̂ T (h) , c.f. (1.2) and Theorem 3.1,we obtain the following bound ( 2 [0; ])TM2 d22 (2 S T ( ); 2 ST ( ))= T 22 inf E0@Xjhj M w( hM ) [f ̂ T (h) (h)g f ̂T (h) (h)g] e ih1A2inf8<:E0@Xjhj M w( hM )e ih T jhjXt=1 1X; =0 f̂ (P )̂ (P )g "t "t+h+h; E ("1)2 1A2+ E0@Xjhj M w( hM )e ih T jhjXt=1 1X; =0"t "t+h "t "t+h+h; E ("1)2 E"211A29=; ;where the in mum is over all i.i.d. random vectors (" t ; "t) ; t 2 Z ; with given marginals"1 F and " 1 F̂ cT :Bound the rst expectation byXjhj;jkj MXs;t X; X; ĵ (P )̂ (P )j ĵ (P )̂ (P )jE"t "t+h "s "s+k+h; +k; E ("1)2 2 :SinceE " i1" i2" i3"i4 =8<: E ("1)4 ; i1 = i2 = i3 = i4(E ("1)2)2 ; two pairs of di erent indices0; otherwise(8.21)25 and sincê (P ) = f̂ (P )(P )g + f (P ) g + f g ;(8.22)cf. (8.6) for the de nition of (P ) , we obtain by a careful computation from Lemma 8.2and Lemma 8.3 that the rst expectation converges to zero in probability.Finally bound the second expectation byXjhj;jkj M T jhj;T jkjXs;t=1 1X; =0 1X; =0 jj f t ;s t+h ;s+k + t ;s+k t+h ;s gn E("1)2 2 2 (E"1"1)2 +E"21 2o + oP (1) :Another careful estimation of the various sums above yields that they are bounded inprobability.SinceE("1)2 2 (E"1"1)2 +E"21 2(jE" 1("1 "1)j + jE"1(" 1 "1)j) OP (1)d2("1; "1) OP (1) = oP (1) ; cf. Proposition 3.1 ,the second expectation is also negligible. This concludes the proof of Theorem 6.4.ReferencesAhlfors, L. (1966). Complex Analysis , 2nd ed.. McGraw-Hill, New-York.Anderson, T.W. (1971). The Statistical Analysis of Time Series. John Wileyand Sons, New York.Baxter, G. (1962). An Asymptotic Result for the nite Predictor. Math. Scand.10, 137-144.Bhattacharya, R.N. and Denker,M. (1990).Asymptotic Statistics. Birkhauser,Basel.Bickel, P. and Buhlmann, P. (1995). Mixing Properties and Functional Cen-tral Limit Theorems for a Sieve Bootstrap in Time Series. Tech. Report, Dept. ofStatistics, Univ. of California, Berkeley.Bickel, P. and Freedman, D. (1981). Some Asymptotic Theory for the Boot-strap. Ann. Statist. 9, 1196-1217.Brockwell, P. and Davis, R.A. (1991). Time Series: Theory and Methods, 2nded., Springer, New York.Buhlmann, P. (1993). The Blockwise Bootstrap in Time Series and EmpiricalProcesses. PhD thesis, ETH Zurich. 26 Buhlmann, P. (1995). Sieve Bootstrap for Time Series. Tech. Report No. 431,Dept. of Statistics, Univ. of California, Berkeley.Buhlmann, P. (1996). Con dence Regions for Trends in Time Series: A Simulta-neous Approach with a Sieve Bootstrap. Tech. Report, Dept. of Statistics, Univ. ofCalifornia, Berkeley.Buhlmann, P. and Kunsch, H.R. (1994). The Blockwise Bootstrap for GeneralParameters of a Stationary Time Series. To appear in Scand. J. Statist. 22 .Dahlhaus, R. and Janas, D. (1996). A Frequency Domain Bootstrap for RatioStatistics in Time Series. Ann. Statist. 24, 1934-1963.Doukhan, P. (1994).Mixing: Properties and Examples. Lecture Notes in Statistics85, Springer, New York.Franke, J. and Hardle,W. (1992).On Bootstrapping Kernel Spectral Estimates.Ann. Statist. 20, 121-145.Ganssler, P. and Stute, W. (1977). Wahrscheinlichkeitstheorie, Springer, Hei-delberg.Hannan, E.J. and Kavalieris, L. (1986). Regression, Autoregression Models. J.Time Ser. Anal. 7, 27-49.Janas, D. (1993a).Bootstrap Procedures for Time Series. Dissertation, UniversitatHeidelberg.Janas, D. (1993b). Edgeworth Expansions for Spectral Mean Estimates with Ap-plications to Whittle Estimates. Ann. Inst. Statist. Math. 46, 667-682.Kreiss, J.-P. (1988). Asymptotical Inference for a Class of Stochastic Processes.Habilitationsschrift, Universitat Hamburg.Kreiss, J.-P. and Franke, J. (1992). Bootstrapping Stationary AutoregressiveMoving Average Models. J. Time Ser. Anal. 13, 297-317.Kunsch, H.R. (1989). The Jackknife and the Bootstrap for General StationaryObservations. Ann. Statist. 17, 1217-1241.Lien, G. (1994). Zeitreihen mit Trend.Dissertation, Technische Universitat Braun-schweig.Mammen, E. (1992). When Does Bootstrap Work? Lecture Notes in Statistics77, Springer, Heidelberg.Masry, E. and Tj stheim, D. (1995). Nonparametric Estimation and Identi -cation of Nonlinear ARCH Time Series. Econometric Theory 11, 258-289.27 Paparoditis, E. and Streitberg, B. (1992). Order Identi cation Statistics inStationary Autoregressive Moving Average Models: Vector Autocorrelations and theBootstrap. J. Time Ser. Anal. 13, 415-434.Rudin, W. (1987). Real and Complex Analysis (3rd ed.).McGraw-Hill, New York.Swanepoel, J.W.H. and van Wyk, J.W.J. (1986). The Bootstrap Applied toPower Spectral Density Function Estimation. Biometrika 73, 135-141.Wu, J. (1986). Jackknife, Bootstrap and other Resampling Methods in RegressionAnalysis. Ann. Statist. 14, 1261-1343.Zelazko, W. (1973). Banach Algebras. Elsevier Publishing Company, Amsterdam.28